Entropy, Inference, and Channel Coding

This article surveys application of convex optimization theory to topics in Information Theory. Topics include optimal robust algorithms for hypothesis testing; a fresh look at the relationships between channel coding and robust hypothesis testing; and the structure of optimal input distributions in channel coding. A key finding is that the optimal distribution achieving channel capacity is typically discrete, and that the distribution achieving an optimal error exponent for rates below capacity is always discrete. We find that the resulting codes significantly out-perform traditional signal constellation schemes such as QAM and PSK. AMS(MOS) subject classifications. Primary: 94A24. 94A13, 94A17, Secondary: 94A34, 94A40, 60F10